refactor(hnsw): use turbo for distance calculation - #735

Summary

This PR starts migrating HNSW distance calculation from the legacy IndexMetric callback mechanism to the Turbo quantizer interface.

The goal is to make quantizers own vector encoding, query preprocessing, distance dispatch, and dequantization, allowing the old converter/reformer and direct metric invocation paths to be removed as the remaining quantizers are migrated.

What changed

• Extend HNSW to accept a Turbo quantizer as its distance provider.
• Route all relevant HNSW distance operations through the quantizer interface:
  • data-to-data distance during graph construction
  • data-to-query distance during search
  • batch data-to-query distance
  • brute-force search distance
• Let the quantizer manage vector and query encoding instead of invoking the legacy converter/reformer pipeline.
• Use the quantizer to dequantize fetched vectors and search results.
• Refresh quantizer-backed distance functions when pooled HNSW contexts are reused.
• Carry quantizers through HNSW merge and reducer operations.
• Persist quantizer metadata so reopened indexes restore the correct distance implementation.
• Detect legacy HNSW indexes without quantizer metadata and reopen them using the old pipeline for backward compatibility.
• Force-link Turbo into the Python extension on Linux, macOS, and Windows so factory registrations are retained.

Current migration scope

The new quantizer-owned distance path is currently enabled for dense FP32 HNSW indexes using:

• L2 or cosine distance
• rotation disabled
• internal vector storage

Other configurations temporarily continue to use the legacy distance path until their Turbo quantizers are integrated.

Motivation

Previously, HNSW distance calculation was spread across converters, reformers, IndexMetric, and streamer-level function callbacks. This made encoding and distance dispatch separate concerns and required HNSW to understand implementation-specific details.

Moving distance calculation behind the Turbo quantizer interface provides one owner for:

• vector representation
• query preprocessing
• single and batch distance kernels
• runtime SIMD dispatch
• vector reconstruction

This simplifies HNSW and establishes a common path for removing the old distance invocation mechanism.

Compatibility

• Existing HNSW indexes without Turbo quantizer metadata continue to open through the legacy converter/reformer path.
• Newly created eligible indexes persist their quantizer metadata and restore the Turbo distance path after reopening.
• Unsupported configurations retain their existing behavior during the migration.